Statistical Inference in Large Antenna Arrays 
under Unknown Noise Pattern 

Julia Vinogradova, Romain Couillet, and Walid Hachem 



Abstract 



' In this article, a general information-plus-noise transmission model is assumed, the receiver end 



o 



(N 



O 



of which is composed of a large number of sensors and is unaware of the noise pattern. For this model. 



and under reasonable assumptions, a set of results is provided for the receiver to perform statistical 
g : eigen-inference on the information pan. In pamcular, we introduce new methods for the detection. 



counting, and the power and subspace estimation of multiple sources composing the information part 
of the transmission. The theoretical performance of some of these techniques is also discussed. An 
f-H I exemplary application of these methods to array processing is then studied in greater detail, leading 



^ . in particular to a novel MUSIC-Uke algorithm assuming unknown noise covariance. 
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SIC algorithm. 

I. Introduction 

A. Motivation 



^ ! Consider the very general information-plus-noise transmission model with multivariate output yt S 

H ■ 

_Cd_. at time t 

yt = Hxt + vt (1) 

where xt G is the vector of transmitted symbols at time t, H € c^x^ is the linear communication 
medium, and vt G the noise experienced by the receiver at time t. 

Array processing consists in a set of tools to perform statistical inference on the information part 
composing yt. The first tool is the mere detection of this information (called then a signal source), 
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that is the question whether K > 0. Once source signals are detected, the next operation consists in 
the evaluation of their number, i.e. estimating K. When the existence of these sources is guaranteed, 
several of their parameters can then be retrieved. One of these parameters is the transmission power of 
the source, or alternatively, the distance from the source to the receiver. Denoting H = [hi, ... , h^], it 
is also of interest to retrieve information from the individual hk vectors. In wireless communications, 
these represent channel beams which the receiver may want to identify in order to decode the entries 
of xt. In array processing, they stand for steering vectors parameterized by the angle-of-arrival of the 
source signals. 

In order to perform these tasks, one assumes the observation of T (non-necessarily independent) 
samples yi, ... ,yT of the process yt. Denoting Yr = T^^/^[yi, . . . , yx], the first mentioned estimators 
are often based on the eigenvalues of YtY^. When it comes to vector identification, the interest is 
rather on the eigenvectors of YrY^. The standard eigen-inference approaches in the literature often 
rely on two strong assumptions: (i) T is large compared to N and (ii) the statistics of vt are partially 
or perfectly known due to independent (information-free) observations of the process vt- This article 
revisits these methods by proposing alternative algorithms to perform eigen-inference for the model 
([T|) accounting for the aforementioned limitations (i) and (ii). 

B. Literature review 

Assuming T ^ oo, N fixed, and vt white Gaussian with known variance, the energy detection 
procedure |IT] allows for the detection of signal sources by evaluating the total received power which 
is compared to a threshold that ensures a maximum false alarm rate. If the signal structure is known, 
the parameters composing H can be recovered from the eigenvalues and eigenvectors of E[yty\^], 
which can be estimated through the sample covariance matrix YtY^, Yt = T^^/^[yi, . . . ,yT] G 
(j-^AfxT rj,^ estimate the number of sources K, the Akaike information criterion (AIC) ||2] and the 
minimum description length (MDL) 131, iffl were historically proposed, which rely on functions of 
the eigenvalues of YtYj^ . The MDL is T-consistent while the AIK tends to overestimate the number 
of sources as T — oo. In terms of power estimation, since YtY^ E[yty\^], a T-consistent 
estimate of the powers is easily obtained by mapping the eigenvalues of YtY^ to those of E[yiy[^]. 
When the vectors /i^ = are steering vectors and one aims at retrieving 6^ for k = 1, . . . , K, 

the multiple signal classification (MUSIC) algorithm Q allows for a T-consistent estimation of the 
angles 9i, ... ,6k by determining the local maxima of the quadratic forms 7(6*) = h{9)^Ilh{9) where 
n is a projector on the eigenspace of E[yty}^] corresponding to its K largest eigenvalues (assuming 
\\h{9)\\ constant with 9). 

Due to the increase of the antenna array sizes and the need for faster detection and estimation 
dynamics, modem antenna array technologies have to deal with the scenario where the condition 
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T ^ N is no longer met. Under this condition, since YrYP becomes a poor estimator for E[yt?/[^], 
most of the above techniques collapse. New methods, based on the field of large dimensional random 
matrix theory, have therefore emerged, which assume that both N and T are large and that the 
ratio N/T is non-trivial. The AIC and MDL algorithms are in particular improved in O using better 
estimators for functional of the eigenvalues of E[yty\^]. In terms of power estimation, A^, T-consistent 
techniques were proposed in [7|. The MUSIC algorithm was improved on the same grounds in [8| 
into the so-called G-MUSIC estimator. 

A second difficulty faced by antenna array technologies is that the interfering environment may be 
far from white Gaussian. The vi, . . . ,vt may not be independent or the spatial correlation of vt may 
not be white. When the noise is not white, the energy detection procedure is not valid as no false 
alarm threshold can be set. When the noise is close-to-white Gaussian with unknown variance, the 
generalized likelihood-ratio test (GLRT) [9 | copes with the indetermination of the variance. Similar 
schemes are analyzed in the large N, T regime in lITOl . ifTTI . |[T2l . |[T3l . If the noise is not white, it is 
difficult to derive any test for detection. The power and direction estimation techniques equally suffer 
from this indetermination, because too little is a priori known of the eigenstructure of Vr^j^ ^^^^ 
Vt = T^^/^[wi, . . . , Vt]- To circumvent this issue, one generally assumes the existence of a sequence 
of T' pure-noise test samples which are used to "whiten" the observations. For T' large compared to 
N, after whitening, the noise becomes white Gaussian with unit variance, leading back to traditional 
schemes. For A^, T' simultaneously large, the whitening procedure gives rise to a noise matrix of the 
F-matrix type l[T4l. ifBl. 

However, the requirement to possess observations purely composed of noise may be impractical in 
real systems. As such, in this article, we address the problems of detection, counting, and parameter 
estimation of multiple sources without resorting to a pre-whitening of the received data matrix Yt- 
Since the problem may not be well-posed in its generality, we assume a set of reasonable conditions: 

m N,T ^ oo, N/T c > 0, K constant. This allows for YtY^ to be seen as a small rank 
perturbation of VtV^. 

• Vt = WtR^'^ {i.e. white in space, correlated in time), where Wt G C^^'^ is standard complex 

1/2 

Gaussian and Rt is a deterministic unknown Hermitian nonnegative, or Vt = R-r {is- 
white in time, correlated in space)|3 

• As N/T — )• c, the eigenvalues of VtV^ tend to cluster in a compact interval. This assumption 
is satisfied by most noise models used in practice, e.g. auto-regressive moving average (ARMA) 
noise processes (see Section IIII-BI ). 

'Assuming the general correlated noise in both time and space would lead to too much indetermination and is so far too 
difficult to address. 
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• The source signals in xt are random, independent and identically distributed (i.i.d.), even though 
this assumption can be relaxed in many cases. 

Under these assumptions, we show that a maximum of K isolated eigenvalues of YtY^ can be 
found for all large N, T beyond the right edge of the limiting eigenvalue distribution support of 
VrVp. This phenomenon is at the origin of the detection and estimation procedures developed in 
this paper. Precisely, we show that the isolated eigenvalues of Ir^j^ uniquely mapped to 

individual signal sources. The presence of these eigenvalues will be used to detect signal sources as 
well as to estimate their number K while their values will be exploited to estimate the source powers. 
The associated eigenvectors will then be used to retrieve information on the vectors /i^. 

The remainder of the article is structured as follows. In Section JIl we introduce the system model 
and recall important results from the random matrix literature. In Section Jill we introduce the source 
detector and parameter estimators for the generic model ([1} and for a specific array processing scenario 
with an ARMA noise process. In Section |IVl we study the second order statistics of some of these 
estimators. Simulations are then provided in Section |V] The article is concluded by Section |Vll Some 
technical lemmas are proved in the appendix. 

Notations: The superscript {■)^ is the Hermitian transpose of a matrix and ||-|| denotes the spectral 

a s P C 

norm. The symbols — >, and — > stand respectively for the almost sure convergence, the 
convergence in probability, and the convergence in law, while "w.p. 1" means "with probability 
one". We denote by M{a,a'^) the real Gaussian distribution with mean a and variance and by 
CM{a,a^) the complex circular Gaussian distribution with mean a and variance a^. We denote by 
Ski the Kronecker delta function {= 1 ii k = £ and otherwise) and by 5x the Dirac measure at x. 

II. Assumptions and known results 

1/2 

Consider a sequence of integers = N{T), T = 1,2,... and matrices Yt = At + WtR^ ^ 
C^^^ where At stands for the signal matrix and Vt = WtR^"^ for the noise matrix]^ We assume 
the following asymptotic regime: 

Assumption \. As T oo, ct = N/T c> 0. 

A. Hypotheses on the noise matrix 

We first characterize the assumptions on Vt — WtRt'^- 

Assumption 2. Wt = T^^^^[wn,t]n't=i> with {wn,t)n,t>i infinite array of independent CAf{0, 1) 
variables. 

^Up to studying instead of Yt, the noise correlation can be eitiier in time or in space. 
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Assumption 3. Rt € C^^-^ is Hermitian nonnegative with eigenvalues afj,, . . . ,a^j, satisfying: 

1) With VT = T-'Y.i =1 ^ct^t' V, a probability measure with support supp(z^) — [ap^hu] C 
R+ = [0,oo). Moreover, i^({0}) = 0. 

2) The distances from the a^j, to supp(i^) satisfy: 

max d ((T?T: supp(z^)) > 0. 

tG{l,...,T} ^ ' T^oo 

Let Ai,T > . . . > Aat^t be the eigenvalues of VrVP = WtRtW^ and let tt = iV"^ Eili ^x,.t 
be its spectral measure. The asymptotic behavior of tt is of prime importance in this paper. We recall 
some well known results describing this behavior (see llT6l . ifTTl for Items [l]-[6l ifTSl for Item|4l and 
|[T9]| for Item Ell : 

Theorem 1. Under Assumptions \2}^ the following hold true: 
1) For any z G C+ = {z G C, > 0}, the equation 

t 



has a unique solution rn G C+. The fiinction m(z) = m so defined on C+ is the Stieltjes 
transform ( ST^ of a probability measure ix. 
2) For every bounded and continuous real function f, 



f{t)TT{dt) ^ / fitMdt) 
T->oo J 

and therefore fi, defined by (O, is the limiting spectral measure of VrV^. 

3) The function 

m(z) = / — ——vidt) 

^ ' J z{l+cm{z)t) ^ ' 

is defined on C+ and is the ST of the probability measure /i = c/i + (1 — c)5o, limiting spectral 
measure ofVpVr- As such, m(z) = cm(z) — (1 — c)/z. 

4) fj, is of the form fj.{dt) = max(0, 1 — c^^)6o + f{t)dt where f{t) is a continuous density on 
(0, oo). The support of f{t)dt is a compact interval [a, 6] C M+, and f{t) > o?i (a,^). 

5) For any interval [xi,X2] C (0, a) U (6, cxd), 

(J{z : Xi^T G [2;i5 2;2]} = w.p. 1 for large T. 

6) The function mj^jx) = N^^ 'l2n=ii^n,T — x)~^ converges w.p. 1 to m(x), and uniformly so on 
the compact subsets of (b, oo). 

^We recall that the ST of a probability measure fi with support in R is defined by m^{z) = J{t — z)^^fi{dt). It is 
analytic on C — supp(/i) and completely characterizes the measure /j.. 



January 3, 2013 



DRAFT 



6 

A procedure for determining the interval [a, b] from the knowledge of c and u is provided in 
lITSl . We are interested here in the determination of the upper bound b, to which Ai^r converges. 
This can be done with the help of the following proposition. Observe that m{z) can be extended to 
C — ({0} U [a, b]) and that m(x) = f{t — x)^^ii{dt), its restriction to M, is negative and increases to 
zero on oo). Recall that supp(z^) = [a,y,6,y] C M+. 

Proposition 1 (see ifTSll ). The point b defined in Theorem [HI?]) coincides with the infimum of the 
fiinction 

x(m) = -— + / - — ^ v(dt) 

^ ^ m J 1 + cmt ^ ' 

on the interval (— (c6j^)~^, 0). On this interval, there is a unique nib (rrih < 0) such that x(m) — )• b 
as m I mf,. The restriction ofx{m) to the interval (mfc,0) coincides with the inverse with respect to 
composition of the restriction of m(x) to {b, oo). 

In order to easily characterize the value of b, it will be convenient to make an assumption on the 
measure u which will not be restrictive in practice: 

Assumption 4. Ifi'{{biy}) = 0, then there exists e > andafunction fu{t) > C{bjy — t) on \by — £,bi,] 
with C > such that for any Borel set A of [ui,, by], 

v{Ar\%-eX\)= I fu{t)dt. 

JAr\[K-eM 

This assumption leads to the following corollary to Proposition [U proven in Appendix [A] 
Corollary 1. Under Assumption |?] 

1 I' t 

b = + / vidt) 

nib J 1 + crriht 

where nib i^ the unique solution in (— (c6iy)~^, 0) to the equation in m 

B. Hypotheses on the signal matrix 

We now turn to the hypotheses on the signal matrix At: 

Assumption 5. Let K > be a fixed integer The matrix At S C^^'^ is random, independent of 
Wt, with rank rank{AT) = K w.p. 1 for all large T. In addition, supj^ \\At\\ < oo w.p. 1. 

In the remainder of the paper, when K < mm(N,T), the notation At = UtB}^ refers to any 
factorization of At where Ut G c^x^ satisfies U^Ut = Ik- By Assumption [51 the rank of 
Bt G C^""^ is equal to K, w.p. 1. 
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Assumption 6. There exists a factorization At = UtB^ such that, for any z € C — supp(i^), 

{Rt - zIt)~^ Bt my{z)P 

T— s>oo 

for some P = diag(pi/jj , . . . ,ptljj, pi > . . . > pt > 0, ji + ■ ■ ■ + jt = K and where it is recalled 
that my{z) is the ST of the probability measure v. 

Remark 1. Assumption |6] is in general very strong. It basically requires that either the right eigen- 
vectors of At or the eigenvectors of Rt be sufficiently isotropic. It is however often met in practice: 
Array Processing: Consider the model At = HtP^^'^S^, with Ht = [h{9i),--- ,h{9K)] (Ok dis- 
tinct) the matrix of steering vectors, P = diag(af , . . . , a|^) the matrix of source powers, and 
St = T^^^'^[s* f,]J'j^-^ the matrix of source signals, and take Vt = WtR^"^. Assume the s^^t 
i.i.d. of zero mean and unit variance and [\/Nh{6)\n = e~^'^*"'^™(^) as in a uniform linear 
array. In this setting, decomposing At = UtB^ with Ut = Ht{HtH^)~^/'^ and Bt = 
StP^''^{HtH}^YI^, we can show {HtH}})~^I^ ^ Ik while S}^{Rt-zIt)-^St ^ my{z)P 
so that Assumption |6] holds. See the proof of Lemma Ujfor details. 
MIMO Communication : Let At = HtP^^^SI^, with now Ht = [hi, ... , hx] the wireless channels 
with i.i.d. zero mean and unit variance entries of K transmitters, P their diagonal power 
matrix, and St their matrix of transmitted zero mean unit variance i.i.d. signals. Taking now 
Vt = R^'^Wt, i.e. spatially correlated noise, and considering Y,p instead of Yt, we may 
write A}} = UtB^ with Ut = St{StS^)-^/^ and Bt = HTP^''^{STS^f/'^ to obtain 
B^{Rt - zIn)~^Bt ^ m^{z)P. 

Section IIII-AI will introduce the main results of the article, and in particular the new detection 
and estimation procedures under the general hypothesis of Assumption [6l Since the results of Sec- 
tion |IIFA1 may be difficult to grasp in the full generality of the set of hypotheses, we will then devote 
Section UlI-BI to the specific study of Item 1) in Remark [T] with vt an ARM A process for which an 
improved MUSIC algorithm to estimate the angles 6k will be proposed. 

C. Results on the information-plus-noise matrix 

Before moving to these applications, we first recall the main results concerning the eigenvalue 
distribution of YtY^. Since Yt^t most a rank 2K perturbation of VtV^ with K fixed, Weyl's 
interlacing inequalities ll20l Th. 4.3.6] show, in conjunction with Theorem [T] that the spectral measure 
of YtY^ also converges to fi in the sense of Theorem [Tl|2l). However, a finite number of eigenvalues 
of It^t might stay isolated away from the support of fi 1QJ[ Th. 2.2]: 
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Theorem 2. Under Assumptions\J}^ let n and [a, b] be as in Theorem\J\ Let Xi^t > • • • > Xm,t 
the eigenvalues ofYTY^ with spectral measure fx = N^'^ Si^i ^i,T- Then: 
1 ) For every bounded and continuous real function f, 



f{t)fT{dt) I f{t)p{dt). 

T->oo J 

2) For any interval [xi,X2] C (0, a) 

tj{i : Xi^T £ [a;ija;2]} = w.p. I for all large T. 

3) The function g(x) = xm{x)m{x) is positive and decreases from g{b^) to zero on (6, oo). If 
pig(6^) < 1, then X\,t b. Otherwise, let s G {l,...,t} be the largest index for which 
Psg{b^) > 1 (see Assumption^. For k = 1, . . . , s, let pk be the unique solution x in (b, oo) of 
PkS{x) = 1. Then, for i = 1, . . . , s and with jo = 0, 



>^jl+■■■+j^-l+l,T, • • • , Aji+...+j,,r " > P 



a.s. 

''ji + ---+js+l,T 



4) The condition PA:g(6'*") > 1 is equivalent to 

with nih the solution in (—{cbu)^^,0) to Equation Q. 

Proof: The first two items in this theorem are proved in 1)211 in a more general setting than 
in this paper. To obtain the last item, observe that g(x) = — f m(x)(l + cm{x)t)^^ u{dt) from the 
definition of m in Theorem [Tl|3]l and recall that m(x) | as x | 6, where mi, is defined in Corollary 

m ■ 

This theorem shows in particular that the number of isolated eigenvalues of YrYp is upper bounded 
by the rank K of At and it reaches this rank if pt is large enough. 

Remark 2. In the white noise setting, i.e. Rt = It (hence, v = p, is the celebrated Marchenko- 
Pastur law, and Equation ^ boils down to pj. > ^/c (see e.g. / I22I/ ). The source detection approaches 
studied in im\l . y f72l/ . y f73l/ rely on this condition. 

III. Source detection and parameter estimation 

We start by stating the results in the general context of Assumptions [T]-[6l We shall then deal more 
specifically with the model of Remark [T}1). 
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A. General results 

Theorem |2] gives the following signal dimension estimator: 

Theorem 3. Under Assumptions\J}^hold true, let s >0 be the largest integer for which Equation @ 
holds. Let 0<e<(ps/6) — 1 (take po = ooj. Given L > K, define 

kt = arg max > 1 + e 

fce{o,...,L} Afc+i,T 

( take Ao,T = oo j. Then, for all T large, w.p. 1, 

kx =ji + ... +js- 

Proof: Writing k = ji + . . . + jg, Items [D and (3]) of Theorem |2] ensure \k,T Ps > b and 
Xe,T ^bfor £ = k + l,...,L. ■ 

Theorem |3] allows in practice to evaluate the number of strong sources when T is large. This 
however requires e to be taken such that e < (ps/b) — 1, a value which is practically not known. An 
empirical approach consists in taking e sufficiently small (but not too small to avoid counting noise 
eigenvalues), see Section jV] Theorem |3] also assumes that the receiver knows an upper bound L on 
K, which is less problematic in practice. 

In the sequel, for i € {1, ... , K}, we let ]C{i) = 1 if 1 < i < ji, /C(i) = 2 if ji + 1 < i < ji + j2, 

. . ., )C{i) = t if ji H \-jt-i + 1 < i < K. The following theorem provides a means for estimating 

consistently pi, . . . ,ps: 

Theorem 4. In the setting of Theorem 13 let 



1 ^ 



N -kx - , Xn,T - X 



9t{x) = mT{x){xcTrhT{x) + ct — 1) 

1 

9T{h,T) 



Pi,T - —i — T' « = 1, • • • , fcr- 



Then 



Pi,T-PK{i) TT^-^O 



T-s-oo 

Proof: Recall that Ai^t > • • • > Aat ^ are the eigenvalues of WtRtW^. In the proof, we restrict 
the elementary events to belong to the probability one set where Ai^r — )• b, rnrp{x) — )• m(x) uniformly 
on the compact subsets of (5, oo) (see Theorem Xi^x — ^ PK{i) for i = 1, . . . , ji + • • • + js, 

Ajj_| hjs+i,T — ^ b, and kx ^ ji + ■ ■ ■ + js from Theorems [T] |2] and |3] Observe that YtY^ is at 

most a (nonnegative) rank 2K perturbation of VrVp. In these conditions, Weyl's inequalities |[20l 
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Th. 4.3.6] ensure A„^t < ^n-2K,T and An,T < \i-2K,t for = 2^ + 1, . . . , A^. For any x > h and T 
large, we then obtain 

1 /^v^^ 1 ^ 1 

= mj^{x) + erix) 



N -kr \ t^. ^n,T -X ^ X J, - X 



where erix) 



uniformly on compact sets of (6, oo), and 

N-2K , AT 

i / 

fflT 




< 



N 

E 



1 " 



^ \n=fcT+l+2ii' 91=^-2/4-+ 



1 A„,T - X 



= + e'rp{x) 

where e^(x) — )• uniformly on compact sets of (6, oo). Consequently, griK^r) — siK,T) for 
i = l,...,kT. Clearly, g{Xi,T) - g(P/c(i)) so that gT{K,T) - g{pic{i)) which, along with 
sipK{i)) = 1/P/C(i)' gives the result. ■ 
Let now At = UtB^ following Assumption [6] and write Ut = [Ui^t, ■ ■ ■ , Ut^r], U^^t £ C^^-'^ 
We introduce the orthogonal projection matrix He^r = Ui^tU^t ^ C^^^ ■ Similarly, we denote 
tlg^T the orthogonal projection matrix on the eigenspace corresponding to the set of eigenvalues 
{Ajj+...+jj_j+i,T, • • • , ^ji+...+je} in YtY^, for £ = 1, . . . ,t (jo = 0). With these notations, we have 
the following estimate of bilinear forms of the type a^He^TbT- 

Theorem 5. Under Assumptions\T\^ let ay, hx G be two sequences of deterministic vectors with 
bounded norms and let }C{i) < s with s the largest integer for which ^ holds. Then: 



-0'T^IC(i),TbT 



-> 0. 



T^oo 



"iT(Ai,T)5r(Ai,T) 

Proof: From Assumption |6l B^Bt P (multiply each side of the convergence by — z and 
take z large). Therefore, pi, . . . ,pt are the limiting positive eigenvalues of AtAI^. For Rt = In, the 
theorem thus coincides with Il22l Theorem 2] since then Vr = Wt is a bi-unitarily invariant (here 
Gaussian) matrix as requested by ||22] Assumption 2]. We now reproduce the steps of Il22l Theorem 2] 
under our set of assumptions. ll22] Equation (8)] remains valid in our setting which, under the present 
notations, reads 

aMn^T^T = -— / a^QJz)bTdz + — I a^Hrizy'^bTdz (5) 
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for Ci^T a complex positively oriented contour enclosing only the eigenvalues Ajj+...+j^_^+i_r, • • • , ^ji+...+ji,T, 
with 

«? = [aI,T,0,...,0], hl = [hlT,Q,...,Q] 

Qt{z) = {VtVP - zIn)-\ Qt{z) = {V^Vt - zIt)'^ 

zQt{z^) VtQt{z^) 
Qt{z^)V^ zQt{z^) 



zU^Qt{z^) 
B^QTiz^)V^ 



zU^Qt{z^) 
B}^Qt{z^)V^ 



Ht{z) 



zU^Qt{z^)Ut U^VtQt(z'^)Bt + Ik 
B^Qt{z'^)V^Ut + Ik zB^Qt{z'^)Bt 
Let i < s. From Theorem |2]-2), for all large T w.p. 1, the first term on the right-hand side of ([5) 
is null (no pole of lies in Ci^t for large T), while in the second term ^ can be replaced by 
a contour Cc enclosing pi but no p^, k i. We must now prove a^IlT{z)hT — a^HT{z)hT 
where 







aT 



zm.{z'^)U}^ 




zm(z2)[/^ 




6t 



Hriz) 



zm{z'^)lK Ik 
Ik zih(z'^)P 



By II2TI Lemmas 4.1-4.6], \\aT — otII —> 0, 



0, 



Ht{z) 



Ik 



Ik 

B'^{lT+cm{z^)RTy^BT 



0. 



Assumption|6]and the definition of m{z) then imply || -^B^^ [It + cm(z^)i?j') ^ BT—z'm{z'^)P\\ 
0, which finally gives a}^HT[z)bT — a\^HT{z)bT — — >• 0. For z G Ci, zm(z^) and zm(z^) are bounded 
by [d(C£,supp(/x))]"^ Take < e < d(C£, supp(/x)). Then, for all large T, zQt{z^) and ^^(z^) 
are bounded by w.p. 1. The dominated convergence theorem therefore ensures that 



0. 



Residue calculus of the right-hand side integrand as in ||22] Equations (lO)-(ll)] then gives 



-aj'Yii^j'bT 



T-j-oo 



■> 0. 
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Take i such that lC{i) = £. Using Xi^t —> Pe, rhT{x) m(x), gxix) —> g,{x), and g'rp{x) 
g'(x) for X outside the support of ^ then concludes the proof. 



B. Narrowband array processing 

We now apply the results of Section ITlI-AI to the array processing model of Remark [T] Consider a 
uniform linear array of N antennas which captures T successive realizations yi , . . . , of the random 
process: 

K 

yt = ^ akh{9k)sk,t + vt (6) 



k=l 



with ai > . . . > qk > the amplitude of sources 1, . . . ,K, h{9) G C the steering- vector function 



h{e) 



1 



he 



-2in sin t 



-2m{N-l)sme 



(7) 



with Ok the angle-of-arrival of the signal from source k (the 9^ are assumed distinct), Sk^t £ C the 
signal emitted by source k at time t such that {st,k)'^il=i is an infinite array of circular complex i.i.d. 
random variables with Esi^i = 0, E|si^ip = 1, and E|si.i|^ < oo, and vt G the noise received 
at the sensor array at time t. 

Denoting Yt = [yi, . . . , yr] G C^^'^, © takes the form 



(8) 



where Ht = /il^a), • • • , /^(^i^)] G C^"^, St = T-^/'[sl G C^><^, P = dmg{al a\), 



andVT = T-^/'^[vi,...,VT] G 



■^NxT 



. We assume the rows of ^/TVt to be independent snapshots of 



a complex Gaussian circular causal ARMA(m, n) stationary process. This process can be represented 
as the output of a filter with transfer function p(2) = (1 + qiz~^ + . . . + amZ^'^) /(I + Pi^^"^ + . . . + 
PnZ'"^) driven by a standard complex Gaussian circular white noise. For \z\ > 1, p(z) = X]^o V'^-^"^ 
where ^ \ipe\ < oo, and we can write Vt = WtRt with Wt as in Assumption |2] and 



Rt 



ro ri 
ri-T ■ ■ ■ 



■ ■ ■ TT-l 

ri 

r-i ro 



with Vk = J2e>o i^f^+k'ipe ^'^^ ^'^y ^ G N, the matrix being nonnegative. 

Lemma 1. Under Assumption \J} the model dH) satisfies Assumptions |2]-[6] with v defined by 



g{t)v{dt)= / y(|p(exp(2z7rn))ndn 




(9) 
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for every positive measurable function g, and with P in Assumption^the matrix of the source powers 

4- 

Proof: We start with Assumptions |3] and |4l If m = n = 0, then = 5i and these assumptions 
are trivially satisfied. Assume min(m, n) > 0. Then Assumption [3HD is a well known result on 
the spectral behavior of large Toeplitz matrices [E3l . Ell- The support of u is the compact interval 
[tty^hy] = [miiiu maxu q{u) = |p(exp(2z7rn))p. It is also well known |[23l §4.2] that 
ay < afrp < by, so that Assumption [SHll) is satisfied. Since p{z) is ARMA, for g{t) the indicator 
function on a set of Lebesgue measure zero, the right hand side of ^ is zero. Hence u has a density 
fy with respect to the Lebesgue measure. Let us provide the expression of fy at a point s G {ay, by) 
such that for any u for which q{u) = s, q'{u) 7^ 0. In a neighborhood of any of these u, q has a 
local inverse that we denote qi Then, for e > small enough, 

dv 



u{s — e,s + e)= / dt = ^ 

Jt:q{t)£[s-e,s+e] u : q(u)=-': 

by the variable change q{t) = v. Letting e 4, 0, we obtain 



siO 28 ^ \q'{u)\ ' 

u : q(u)=s 

This proves fy{s) 00 as s 'I by, implying Assumption |4l 

We now turn to Assumptions |5] and |6] Since the 9i are distinct (modulo vr), H^Ht — ?• Ik- By 
the law of large numbers, S^St — — — Ik- Hence rank(j4j') = K w.p. 1 for all large T, and 
supj^II^tII < 00 w.p. 1. Let us write At = UtB)^ where Ut = IIt{II^Ht)^^^'^ and where 
Bt = STP^/^{H}j\HTy/'^. By HH Lemma 2.7] and < 00, for any z e C+ and any 



4 



4 _C 

— 



for some C > 0. By Markov's inequality, the argument of E| • | converges to zero w.p. 1, and 
this convergence can be extended to C — supp(^). Since T^^ Tr[(i?T — zIt)~^] — ^ rny{z) for 
z G C — supp(i^). Assumption [6] is satisfied. ■ 
With these results. Lemma [T] and Theorems |3] and 5] lead to the following inference methods: 

Proposition 2. Consider the model Let k >0 be the largest integer for which (take ao = 00) 

4> ( [ 1 ,^ TTHT dv^ (10) 

^ \Jo l + cmfe|p(exp(2z7rM))|2 ) 
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with mi, £ (— (c max„ |p(exp(2z7rM))p) ^,0) the solution of 

/ m|p(exp(2CTn))p V _ 1 
7o \l + C'm\p{e-K.p{2i7ru))\'^ J c' 

Given L > K and e > 0, define (with Ao,t = ooj 

kt = arg max > 1 + e. 

mG{0,...,L} Am,+1,T 

Then kx = k w.p. 1 for all large T and e small enough. Moreover, for i = 1, . . . /kx let afj. = 
(5T(Ai,T)) ""^ with gxi^i^T) in Theorem^ Then 

„2 a.s. 2 

«i,r a* • 

Based on Theorem [51 we now provide a source localization method based on MUSIC 121. Recall 
that MUSIC exploits the fact that h{9i)^{Iisi — j)h{6i) = with 11^ ^ a projector on the subspace 
generated by h{9i), . . . , h{9i) for any i < £ < K. Since ||/i(^)|| = 1, 6i, ... ,9i are the arguments of 
the local maxima of 

j^{9) = h{9)^U{Th{9). 

Proposition 3. Let k and kx be as in Proposition^ and denote ui,Ti • • • i^^^ t eigenvectors of 
YtY^ with respective eigenvalues Xi^t, ■ ■ ■ , j,. Then, for 9 G [— 7r/2, 7r/2], 

4(9) -4- (9)^0 

where 

-f^{9) ^ h{9)^UlTh{9) 

mT{Xj,T)gT{Xj,T) 

Proof: Lemma [T] ensures that Assumptions [l]-l6] are satisfied, so Theorem [5] can be applied for 
each i < k. Taking ax = bx = h{9) and Ut = Ht{H}^Ht)~^^'^ as in Theorem |5] we obtain the 
desired result for C/rJC/H, J = diag(4,0), instead of H^^. As {H^HTy^f^ J{H}}Ht)-^I'^ ^ J 
and HtJH^ is the same projector as nf'y, we have h{9)^U'l j,h{9) - h{9)^UTJU^h{9) ^ 0, 
completing the proof. ■ 
Proposition [3] ensures that ^^{9) is a consistent estimator of the locahzation function 7^(^). The 
alternative MUSIC algorithm we therefore propose consists in estimating 9i, . . . ,9^ as the arguments 
of the kx highest maxima of {9). Observe that, although the system models differ in both articles, 
the MUSIC estimator proposed here exactly corresponds to that provided in Il22l . This remark would 
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not hold if it were not for Assumption |6] 

IV. Second order performance analysis 

In this section, we discuss the asymptotic (second order) performance of the detection and es- 
timation schemes derived in Section JII] The model of Section IIII-BI is considered. Following the 
notations of Section Ull-Ai we gather the source powers a| in groups of equal powers pi > ... > pt 
with respective multiplicities ji, ■ ■ ■ ,jt- 

A. Main results 

We start by studying the fluctuations of the isolated eigenvalues of YtY^ . Recall the definition of 
ut in Assumption [3] and recall that ct = N/T. Replacing v and c with vt and ct, respectively, in 
Theorem [T] we obtain that 

mT{z) =(-z+ [ ., Mdt)] (11) 

V J 1 + CTmT{z)t J 

uniquely defines the ST mT{z) of a probability measure supported by IR+. In addition, /i^ 
converges weakly to as T — oo; the Hausdorff distance between the supports of these two measures 
converges to zero ifTTI . |[T9l and, for each h' > b, mT{z) is analytic on C — [0, h'] for all large T. Let 

/—I —1 
— — -——VT{dt) = — Tr(/T + CTmT{z)RT)~^ . 
z{l + CTmT[z)tj zT 

Similarly to Theorem [T]|3]l, rhT{z) satisfies mT{z) = CTmT{z) — (1 — ct)/ z. Consequently, for all 
T large, grix) = xmT{x)mT{x) is defined on (6', oo), b' > b, and, for any k such that PkS{b^) > 1, 
Pk9T{x) = 1 has a unique solution p^^T in oo). 

The main result of this section (Theorem |6]l describes the fluctuations of Xi^t — PK{i),T' i < s, with 
s the largest integer satisfying (|4]i. 

Lemma 2. Consider the model (HJ. Then the function 

A{x) = 1 - c [ ( -J^^^]\idt) 
^ ' J \l + cm{x)tj ^ ^ 

is defined and positive on (b, oo). Furthermore, A{x) as x ^ b and A(x) — )• 1 a5 x — )■ oo. 

Proof: See Appendix iBl ■ 

Theorem 6. Consider ^ with the assumptions of Section \III-B\ Assume in addition E[s" ^(s^ ^y] = 
for u + V < 4 and u ^ v, and let k = E|si^i|^ — 2. Let s be the largest integer (assumed > 1) 
for which (1101 ) holds. For k = 1, . . . , s and all T large, let p^ rp be the unique solution in (b, oo) of 
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PkOrix) = 1. Define (with jo = Oj 











1 


\ 


Vk,T = Vt 






- Pk,T 








I 






1 


/ 



+ c 

(l+cm(pfc)t)2 
Pfcm(pfc) 



(l + cm(pfc)i)2 
Pfcm(pfc)t 



u{dt) 



1 + cm(pfc)t)2 
v{dt), and 

2 



v{dt) 



v{dt) 



1 + cm(pfc)t 

Let Ml, . . . , Ms, Mk = [-M^ m fc]i<^ „i<j^,, be random independent Hermitian matrices such that 
{Mi^rn,k}e<m are independent, Mg^e^k ~ AA(0, afc + + and Mg^m^k ~ C7V(0, + /3fc) /or 

1 < i < m < jk- Let Xk be the M^''— valued vector of the decreasingly ordered eigenvalues of 
{Pkg' {pk))~^ Mk. Then 



(m,T, ■ ■ ■,Vs,t) 



T-s>oo 



> (Xi, • • ■,Xs)- 



Proof: The proof of the theorem is given in Section IIV-BI ■ 
Theorem |6] shows that, after appropriate centering and scaling, the vector of the isolated eigenvalues 
of YtY^ that converge to pk > b tends to fluctuate like the eigenvalues of a certain Hermitian 
matrix with Gaussian elements. If k = 0, this matrix is a scaled Gaussian Unitary Ensemble (GUE) 
matnx Swhen K = 0, sT'^/^iXi^T-br) converges in law to the Tracy-Widom probability distribution 
TW(-), where bx is the finite horizon equivalent to b and s is a scaling parameter that depends 
on c and u |[25l . This result can be generalized to show that for any fixed integer r, the vector 
T^/'^(Ai^T — &Ti • • • 1 K,T — br) converges in distribution to a multidimensional version of the Tracy- 
Widom law. These results and Theorem |6] can then be used to evaluate the error probabilities of the 
source detection schemes described in Theorem |3] and Proposition |2l 

Remark 3. We note without proof that for the specific ARMA model considered here, the measure 
vt can be freely replaced with v in Equation diiP . Tlie error incurred on ra^^z) by this replacement 
is negligible in the ARMA context. 



*We recall that a GUE matrix is a random Hermitian matrix M = [M,j] such that Ma ~ A/'(0, 1), M^j ~ CA/'(0, 1) for 
i < j, these random variables being independent. 
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Theorem |6] can also be used to characterize the fluctuations of the source power estimates: 

Theorem 7. Consider the setup of Theorem^and let pix = {gT{K,T))~^ for i = 1, . . . + - ■ ■+js- 
For k = 1, . . . , s, define (with jo = 0) 



I 



Pjl+-+ 






1 






-Pk 






■+jk,T 




1 



Let Ml;, be defined as in Theorem^and let Xk be the W'' —valued vector of the decreasingly ordered 
eigenvalues of pkM^. Then 

(6,T, • • • , 6,t) > {Xl, ■■■,Xs)- 

1 — ^CXD 

Proof: A sketch of the proof is provided in Appendix O ■ 
As a corollary of Theorem |7] the following proposition provides the behavior of the power estimates 
for extreme values of pk, i.e. for p^ ^ oo and for pk close to the detectabiUty limit given by (fTOl) : 

Proposition 4. Consider the setting of Theorem Let be the infimum of the p^ satisfying (1101 ). 
Mfc be defined as in Theorem |6] and ipk — ctk + Pk + i^'t'k, i^k — ctk + Pk- Then 



Pkipii 



oo, ll^k 



Pkipn 



-)• OO 



Tpk > 1 + K, lpk~ 



> 1. 



Proof: See Appendix iDl 



B. Proof of Theorem |6| 

The proof relies on two ingredients: an adaption of lIlTl Th. 2.3] and a result on fluctuations of 
quadratic forms. Let At = UrBt^ with Ut = Ht{H}^Ht)-^/'^ and Bt = StP^/'^{H^HtYI'^ = 
[Bi^T, ■ ■ ■ , Bt^r], Bk x £ C'^^^''. In |21], it is shown that the r]^ ^ fluctuate like the ordered eigenvalues 
of the matrices {pkg{pkyy^{^/aj^Gk+VfFk,T) where Fk,T = mT{pk,T)B]^j,{lT+CTmT{pk,T)RTy^ 
Bk,T + Ijk ^11^ the Gk are GUE matrices independent of the Fk^T- Using H!^Ht Ik, the law of 
large numbers and the definition of pk^x informally give 

Fk,T ^ [^T{Pk,T){lT + CTmT{pk,T)RT)~^] + l) Ij^, = 0. 

We thus need to study the fluctuations of VTFk^r, which is the purpose of the following lemmas, 
proved in Appendices El IB and iGl respectively: 



Lemma 3. Let Dt G C^^^ be a sequence of deterministic Hermitian matrices with supj. \\Dt\\ < oo. 
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Assume that 

^TVD^- ,p and lTr(diag(Z)T))'- > ^. 

1 T^oQ 1 T— )-oo 

Consider the matrices St defined by ((S]). Then 

Vf(s^DTST - ^Ik) G 

\ 1 / T-j-oo 

where G = [Gij\Kij<K is random Hermitian such that {Gjj}j<j are independent, Ga ~ J\f{0,f3 + 
K(f)) forl<i<K, and Gij ~ CJ\f{0, (3) for 1 < i < j < K. 

Lemma 4, Let 1 < k < s and 

Dt = PkinT{pk,T){lT + CTmT{pk,T)RT)~^- 

Then limsupj' \\Dt\\ < oo, 

^ Tr(I)2 ) , /3fc , and ^ Tv{dmg{DT)f > <Afc 

where ^'^^ 4'k '^re given in Theorem^ 

Lemma 5. Let Mi,..., Mt, Mk = [Mt ,m,fc]i<£,m<jfc> be random independent Hermitian matrices 
such that the {Mi^m,k}e<m are independent, M^^^^k ~ A/'(0, + K(j)k), and M^^m,k ~ CAA(0, f5k) for 
1 < i < m < jk- Then 

{VTFk,T)k=i,...,t {Mk)k=i,...,t. 

The proof ends with the adapted statement of ||2T1 Th. 2.3] jfl 

Proposition 5. In the setting of Theorem^ let Gi, . . . ,Gs, Gk G C-'"^-^'", be independent GUE 
matrices. Then, for any bounded and continuous f : R-^^^* — t- M, 

E[/(7?i,t,...,%t)]-E[/(Ci,...,6)]^0 

where Qk is the random vector of the decreasingly ordered eigenvalues of {pkS{Pky)~^i\/^Gk + 
VTFk,T). 

V. Simulation results 

We consider the setting of Section IIII-BI with signals st^k drawn from a QPSK constellation for 
which K = —1 and K = 1. The signal power af defines the signal-to-noise ratio (SNR). The noise 

^ In fact, |2I, Th. 2.3] characterizes the asymptotic fluctuations of the random variables \/r(Ai,T — instead of the 

y/T{Xi^T — PK{i),T)^ so that the speed of convergence of ut towards v and of ct towards c had to be controlled through 
1211 Assumption 7]. By replacing /o^ with pk,T, the proof of L2ll Th. 2.3] goes on without the need for that assumption. 
Replacing pk by pk^r is enough for the present purpose. 
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is issued from an autoregressive (AR) process of order 1 and parameter o, so that [-Rr]fc,z = o 



k-l 



All other parameters are given in the figure captions. 

In Figure [T] the false alarm rate (FAR) and correct detection rate (CDR) performance of the 
detector proposed in Proposition |2] (consisting in estimating A;t = 1 among 0, . . . , L) is compared 
against the MDL and AIC detectors (consisting also in finding exactly one source). We observe that 
the proposed detector uniformly outperforms the MDL and the AIC detectors, consistently with the 
known inappropriateness of the latter. Note that the AIC particularly fails to detect any source, in 
spite of N growing, demonstrating the inherent inconsistency of this estimator. 

In Figure |2j the receiver cooperation characteristics (ROC), parameterized by e, for different values 
of a are depicted. We compare here our proposed detection scheme against an oracle method which 
assumes perfect knowledge of Rt that is used to whiten Yt before applying the proposed schemes in 
the white noise case. We observe that the proposed detector deteriorates with growing a, which can 
be explained by the natural spread of supp(^) with a large, implying larger inter-eigenvalue spacings 
within the noise subspace and therefore reduced efficiency of the detection test. On the opposite, the 
oracle estimator benefits from increased values of a, due to the SNR gain obtained by the whitening 
procedure. Observe that both approaches perform identically for a = 0, which is expected since the 
system models in both cases are identical. 

Figure [3] depicts the normalized mean square error (NMSE) ]E[(af — af)^a^^] of the power 
estimation of Proposition |2] against its theoretical value obtained from Theorem |6] For the purpose 
of analysis, we assume that the source is always detected, i.e. Ut = 1, irrespective of the SNR. As 
confirmed by Proposition |4l the theoretical variance diverges as | p\im- We however observe that 
in the finite N,T regime, the power estimator errors remain bounded at low SNR. This is explained 
by the fact that, while the theoretical error diverges due to A | (see Lemma |2]) as i PWm, its 
estimator for each A^, T (obtained by replacing m by -mT) is always non-zero even for = pi^rn- In 
the high SNR regime, here with k = — 1, the NMSE becomes linear (in dB scale) with slope —10 
dB/decade. It is easily shown that the limiting SNR gap between the proposed and oracle estimators 
is exactly 



which is merely due to a gain in SNR after whitening. In particular, the larger the correlation parameter 
a, the bigger the limiting gap. 

In Figure |4l the mean square error E[(7(0i) — 7(^1))^] of the localization function at position 
61 = 10° is compared against the performances of the oracle estimator (which performs pre-whitening 
prior to using the estimator of Il22l or equivalently that of Proposition |3j and of the traditional 
MUSIC estimator with localization function 7trad,r(^) — X]fe=i h{9)^ Uk.rUk T^i^) notations 
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of Proposition |2] The source is again supposed always detected so that kx = 1 throughout the 
experiment. The proposed estimator outperforms greatly the traditional MUSIC approach here, which 
is both due to the large N, T regime improvement and to the consideration of the non-white noise 
setting. The oracle estimator shows a huge performance improvement in the low SNR regime, which 
translates the fact that condition dU (which needs to be fulfilled for either method to be valid) is 
extremely demanding when a = 0.6 (due to supp(^) being large). In the large SNR regime, a constant 
gap is maintained which, although we do not provide theoretical support, appears as a similar SNR-gap 
phenomenon as observed in Figure |3] 

In Figure [5j we now take K = 2 sources, with oi = a2 the amplitude of which define the 
SNR, and again assuming kx = 2. Here are compared the performances of resolution of two close 
sources located at 9i = 10° and 02 = 12° for the localization method proposed in Proposition |3] 
for the oracle estimator, and for the traditional MUSIC estimator. The figure of merit, referred to as 
resolution probability, is the probability of identifying exactly two local minima of the localization 
function in the window [5°, 17°]. We observe that the proposed algorithm performs significantly better 
than the traditional MUSIC method, confirming the results of 11221 for the current model. 




12 14 16 18 20 22 24 26 28 30 

N 

Figure 1. CDR (plain curve) and FAR (dashed curves) versus iV with K = 1, SNR= 10 dB, L = 5, £ = 0.75, ct = 0.5, 
and a = 0.6. 
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FAR 



Figure 2. ROC curves with K = 1, SNR= 2 dB, L = 5, iV = 20, and ct = 0.5. 
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SNR (dB) 

Figure 5. Resolution probability versus SNR with K — 2, N = 20, ct — 0.2, and a — 0.6. 

VI. Conclusion and research prospects 

This article introduced a novel set of statistical inference methods for large dimensional information- 
plus-noise models with multiple sources and unknown colored noise. These techniques were proved 
consistent in the limiting regime where both the system size and the number of observations go large. 
The approach pursued here relies on the asymptotic spectral separation between noise and signal in 
the observed sample covariance matrix. Under the same hypotheses, using instead prior information 
on the noise structure, an alternative approach could consist in estimating the noise covariance in the 
presence of signals, similar to [26] which treats the noise-only case. It is expected that this approach 
performs better in the low SNR regime, resurrecting signals unseen by our current method. In the high 
SNR regime, the covariance estimation will instead be too degraded for this method to be beneficial. 
A trade-off is therefore expected between both approaches, which we shall study in a future work. 

In the specific problem of signal detection, the choice of the eigenvalue "gap parameter" e does 
not account for the observation of the small eigenvalues of YtY^ as for the power and direction-of- 
arrival estimation techniques (through mj-). It seems nonetheless natural to be able to evaluate the 
right-edge of supp(/x) from these eigenvalues, thus resulting in a test to compare Xi^T, i = I, ■ ■ ■ , L, 
to the estimated edge. To finely tune the test, one can then use the results from ll25l which proves 
Tracy-Widom fluctuations at the edge with scaling coefficient x"{mb) {rrih given by Corollary [T}. 
However, estimating both the edge and this coefficient constitute a challenging problem so far. 

Appendix 

A. Proof of Corollary [7] 
The derivative 

x'(m) = ^ - c / f - — ) ly(dt) 
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of x(m) is continuous and increasing on {—{cb,^)^^, 0), and x'(m) — )• oo as m t 0. To establish the 
proposition, it will be enough to show that x.'{m)i — )• — oo as m | —{cb,^)^^. This is obvious when 
^{b^) > 0. Assume then u{b^) = 0. When m l —{cbiy)^^, by the monotone convergence theorem 

f f f bH^ 

/ 7 ^^(dt) t / 7 77T^ (dt) > / t; -7^ fJt)dt = oo 

from the behavior of fv{t) near which proves the result. 

B. Proof of Lemma^ 

Considering Equation Q, we obtain after some calculus that m'(x) = T[a?{x)/A.{x) on (6, oo). 
Since m(a;) is negative and increasing on (6, oo), both m'(x) and m^{x) are positive on this interval 
so that A(x) > on (6, oo). 

Proposition [T] shows that b coincides with the minimum of x(m) on ((— c6i,)^^, 0). Moreover, when 
Assumption H] is satisfied (which is the case for the model dSjl by Lemma [Til, the proof of Corollary [T] 
shows that x(m) attains its minimum at a unique point m;, G {{—cby)^^, 0), and x'(mb) = 0. Finally, 
Proposition [T] shows that x(m) is the inverse of m(x) on (6, oo). It results that m(x) — and 
m'(2;) = l/x'(m(a;)) — oo as x | 6. This proves A(x) — )• as x | 6. 

When X — )• oo, both (xm(x))2 = (/ x{t — x)~^ ^{dt))"^ and x'^m^x) = J x'^ {t — x)'"^ ij.{dt) converge 
to 1. Hence, A(x) = {xm{x))'^ {x'^m' (x))^^ — 1, concluding the proof. 

C. Theorem main steps of the proof 

For simplicity, we focus on the fluctuations of VT{pi^t — Pi)- Recall that pi ^ = 9t(Ai,t)^ and 
Pi = 9t{pi,t)^^- Define grp{x) = mj.(x)(xcTl2l7-(a;) + cr — 1) with m-ri^) defined in Theorem [T]|6). 
We have 

= VT{gT{\i,T)-^ - g_^(Xi,T)-^) 

+ vf{gT{Xi,Tr^ - gT(.pi,Tr^) 

— /i,t(Ai,t) + /2,t(Ai,t) + /3,t(Ai,t)- 

As Xi^T > pl, we can replace /i,t(Ai,t) by /i,t(Ai,t)1/(Ai,t) where 1/ is the indicator function 
on a small compact interval / in a neighborhood of pi. Mimicking the proof of Theorem HI we can 
show that sup^jg/ fi,T{x) — > 0. We similarly restrict /2,t to /. On this set, it is possible to show 
that the random process T(my(x) — mT{x)) valued in the set C{I) of the continuous functions 
on /, converges in distribution towards a Gaussian process in C{I). This result was shown in ll27l 
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for I a compact path of C+; this can be generaUzed to the interval / of interest in this proof by 
using the Gaussian tools used in e.g. [21]. As a result, sup^.g/ /2,t(3;) — > 0. To deal with /3 T, 
we start by observing that gT{pk,T) g(Pfc) and {1/ gT{pk,T))' -g'{pk)/g'^(.Pk) = -pls'ipk)- 
Using the result of Theorem |6] and applying the Delta method ED Prop. 6.1.6], we can show that 
/3,t(Ai,t) Pi[Mi]ii. The generalization to the vectors ^k,T defined in the theorem shows no major 
difficulty. 

D. Proof of Proposition |4| 

From Theorem |2] pk i b as pk i pum- Hence, by Lemma |2l A(pfc) — ^ as | pum- Moreover, 
the proof of this lemma shows that |m(pfe)| remains bounded as | b. Hence, since u ^ 60 hy 
Assumption |3l the integrals in the expression of ak are lower bounded by a positive number as 
Pk i Plim- Thus, ttk — > 00 which proves the first part of the lemma. 

When — )• 00, Pk/Pk — ^ 1 and pfcm(pfc) — )• —1. Taking — )• 00 into the expressions of the 
integrals on the right hand sides of the expressions of Uk, and (fik and recalling that ^{pk) — ^ 1> 
we get afe — > 0, /3a; — > 1, and (pk ^ 1, which proves the lemma. 

E. Lemma \3\ sketch of the proof 

The fluctuations of quadratic forms of the type s^Dtst where st G has i.i.d. entries have 
been well studied (e.g. ||29l Th. 2.1], |30, Th. 3]). Here, the vector st is replaced by the matrix 
St G C^^^ which introduces some differences in the proof. We follow here the lines of the proof 
of ll30l Th. 3] and stress the main differences. 

Let VTS^ = [si,--- ,st] where St = [s^*!, . . . , s*^]"^ and let C = [cij] G C^x^ Hermitian 
matrix. Showing that 

and invoking the Cramer- Wold device establishes the lemma. 

Consider the sequence of increasing cr-fields Tt = ^(si, . . . ,S(), t = 1, . . . ,T, and denote the 
expectation conditional to Tt. Then, with Eq = E, 

Vr Tr C (^S^DtSt - ^ Tr DtIr) =VtY^ (Et+i - Ef) Tr CS^DtSt 

t=o 

which is a sum of martingale increments, so that the key tool for establishing Lemma |3] is martingale 
CLT [31, Th. 35.12]. Writing Zt = (Et+i -Et)Ti CSI^DtSt, we need to show: 
• Lyapunov 's condition : there exists 5 > for which 

2.1+5/2 y ^72+5 

t=o 
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The following convergence holds 

T-l 



rV EtZ^ /3Tr(C2) + KaTV[(diag(C))2]. 

^ — ' T->-oo 
i=0 

Taking (5 = 2 and mimicking the calculus of Il30l page 5058] (based on Burkholder's inequality and 

E|si,i|S < oo) gives Xlfjo^ IE[|(IEt+i - '^t)\S)^DTST\j\^\ ^ 0, 1 < i,j < K, which proves 
Lyapunov's condition. Denoting Dt = [djj], 

K t 

TZt = dt+i^t+i Tr C(st+is[^+i - Ik) + 2?fi(^^ Cij ^ slj st+i,i dk,t+i^ ■ 

Using the independence of the Sij and the moments Esi^i = 0, E|si i|^ = 1, and E[s^^(s* i)""] = 
for u V, we obtain 

K K t 

k=l i,j,n=l k,i=l 

Letting Dt = [dijli>j], we have 

T-l K 

T EtZ? =(Tr C^ + ^Yl T '^(diag(i^T))' + ^ Tr CS^^^^t^tC. 

t=o k=l 

Using im Lemma 2.7] and (301 Lemma 3] (or [32. P. 278]), we then get 

^ Tr C5'^£)^Z)t5'tC - Tr C^;^ Tr dI^Dt — ^ 0. 
We finally get the result by observing that 

F. Proof of Lemma ^ 

II2TI Lemma 3.1] shows that for any compact K cM. — supp(//), there exists C > such that 

Vr large, Vt e supp(z^T), inf |1 + CTmT{x)t\ > C 

and hence liminfT'infjgsupp(j,^) |1 + CT"iT(/Ofc,r)i| > 0. It results that limsup^^ II-^tII < 00. Further- 
more, since 



T J (1 + CTmT{pk,T)ty 



the first convergence in the statement of Lemma 51 holds true. 

As for the second convergence, recall that Rt = [rt-n\i<t,n<T, with \rt\ < 00, and define the 
ToepUtz matrix Tt = ['^t-n]i<t,n<T where 7^ = (5^ + cm(pfc)r^ Observe that 1)^ = Pk'mT{pk,T)^T^ ■ 
Let [-Jt- be the modulo-T operator, and let Tt = [7[t-n]r]i<<,n<r be a circulant matrix associated 
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with r^. By lIlTl Lemma 3.1] again, liminfT inf^gp^^] (1 + CTW.T(Pfc,T)|p(exp(2^7r^i))p) > 0, hence 
supj^llrTll < oo. It results that r^^Hr^;^ - Tr^llfro 0' with || • ||fro the Frobenius norm ||23] 
Th. 5.2]. On the other hand, since is circulant, its eigenvector matrix is the Fourier T xT matrix, 
so that we can show 



The lemma is obtained by combining these last two results. 
G. Proof of Lemma \5\ 

We essentially show that we can replace the Bj^ x by y/pkSk,T with St = [Si,t, ■ ■ ■ , St.r], similar to 
Bt- Since 9i ^ 6j if i j, from the definition of the vector function a{9), we have [H}^HT]k,e—SM = 
aT{0k)^aT{0i) - Ski = 0{l/T). Hence, {H^^.Hrf/'^ ^ Ij^ + Et where ||^t|| = 0{l/T). Given 
any sequence Dt of deterministic matrices such that supj^ ll^rll < oo> it can be seen by a moment 
derivation with respect to the law of St that E\[B^DtBt - P^/'^ S)^DtStP^'%/\ = 0{l/T) for 
any k,£<K. Hence, by Markov's inequality, VT{BI^DtBt - P^^'^SI^DtStP^^'^) A 0. Replacing 
Dt with any of the matrices PkmT{pk,T){lT + CT'mT{pk,T)RT)^^ , we get from Lemma |4] that 
supT \\Dt\\ < oo. Therefore, the Bk,T can be replaced with the ^/pkSk^T- The result is then obtained 
upon applying Lemmas [3] and |4] and recalling that, for A; = 1, . . . , t, the Sk,T are independent. 
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